Search for: All records

Abstract Motivated by the residual type neural networks (ResNet), this paper studies optimal control problems constrained by a non-smooth integral equation associated to a fractional differential equation. Such non-smooth equations, for instance, arise in the continuous representation of fractional deep neural networks (DNNs). Here the underlying non-differentiable function is the ReLU or max function. The control enters in a nonlinear and multiplicative manner and we additionally impose control constraints. Because of the presence of the non-differentiable mapping, the application of standard adjoint calculus is excluded. We derive strong stationary conditions by relying on the limited differentiability properties of the non-smooth map. While traditional approaches smoothen the non-differentiable function, no such smoothness is retained in our final strong stationarity system. Thus, this work also closes a gap which currently exists in continuous neural networks with ReLU type activation function. 

Abstract We consider optimization problems in the fractional order Sobolev spaces with sparsity promoting objective functionals containingL^p-pseudonorms, $p\in (0,1)$. Existence of solutions is proven. By means of a smoothing scheme, we obtain first-order optimality conditions, which contain an equation with the fractional Laplace operator. An algorithm based on this smoothing scheme is developed. Weak limit points of iterates are shown to satisfy a stationarity system that is slightly weaker than that given by the necessary condition.